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ABSTRACT 



'’’he Rurdue experiment in mass instruction in 



undergraduate sociology was initiated in the tall of 1967. Lectures 
are combined with group discussions, and achievement in the course is 
evaluated mainly by essays written during the semester. Data reported 
were collected during three successive semesters when minor changes 
were made in course structure. Scores on the general sociology test 
indicated that students were learning a satisfactory amount of basic 
sociology. Questionnaires were used to assess student reactions to 
essays as the basis of grades, to the discussion meetings, to their 
assigned tutor (discussion leader) and to the lectures. Associations 
between these variables were studied, and correlations between 
over-all course evaluation and other response and personal variables 
were investigated while controlling on grade, approval of essay 
system, sex, aqe and semester in college. Findings were that the 
over-all evaluation was positive, that modifications in course 
structure had little effect on student responses, that the general 
reaction to the course was more strongly associated with reaction to 
lectures than to other course structure variables^ and that freshman 
students appear to have special difficulties. (EB) 
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A CONTINUING EXPERIMENT IN MASS EDUCATION— 
A PROGRESS REPORT 



Introduction 

This paper represents a progress report on an on-going program of 
research on Instruction in undergraduate sociology. The Purdue Experiment 
in Mass Instruction was inaugurated in the fall of 1967 and has been contin- 
ued to date. The Experiment is an attempt to satisfy undergraduate needs 
for personalization, among other things, while maintaining some fairly high 
level of instructional quality under conditions of mass enrollment. A 
previous report described the first semester's experience with the course.^ 

The current effort describes the major outlines of the instructional program 
for the past three semesters of the experiment and the responses of the 
students during that period. 

The basic structure of the Experiment in Mass Education is grounded in 
the current need for very large classes. Introductory sociology at Purdue 
enrolls approximately one thousand freshmen and sophomores per semester. On 
the rationale that classes of this size are becoming sufficiently common as 
to constitute a modal type for early undergraduate experience, at least at 
state-supported institutions, and that the problems of such instruction are 
not even clearly understood today, much less well resolved, the Department 
of Sociology at Purdue has embarked upon a long range program of research 
on the problems of pedagogy in the introductory course. The fundamental 
goal of the experiment is the discovery of empirically validated means of 

^eece McGee and Butler Crittenden, "Freshman Sociology at Purdue: An 

Experiment in Mass Education," Purdue Experiment in Mass Instruction, Bulletin 
Number 1, Institute for the Study of Social Change, Purdue University. 
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teaching mass classes with both intellectual satisfaction and efficiency. 

To this end, students in the course are required to complete a personal 
history questionnaire at the beginning of each semester and a major course 
evaluation form at the end of the term. Both questionnaire forms also con- 
tain a set of personality scales and a "general sociology" test as a before- 
after measure of student performance. Additional information, such as SAT 
scores and records of each student's graded progress, is gathered. This 
body of data constitutes the basis for assessing changes in the structure 
and content of the course and for plotting the varying degrees of success 
of the Experiment. 

Basic Format and Semester Modifications 

The general structure with which the course was launched in the fall of 
1967 has been more or less maintained. Offered in the familiar lecture- 
discussion style (with two lectures and one discussion per week), the course 
employs a senior faculty member as lecturer and course co-ordinator and 
twelve graduate teaching assistants, eleven of whom each meet four discus- 
sion sections per week (the twelfth acts as administrative assistant for the 
course). While there is nothing particularly unique about this format, the 
basic focus of the course differentiates it from most other introductory 
programs. Rather than asking students to master facts and data, the goal 
of this program is to teach students to think analytically using sociological 
concepts, in other words to understand and use the sociological frame of 
reference in ways that are meaningful for them. As a result, the conventional 
testing system of exams and term papers was discarded in favor of a system 
of evaluation based on brief weekly or biweekly papers which could reflect 
the kind of learning toward which the course was directed. The topics of 
these "think pieces" ranged from analyses of personal situations based on 
particular sets of sociological concepts to analyses and integration of 
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particular sociological theories. In addition, all attendance requirements 
were dropped leaving the papers as the primary basis of the grade. 

During the first semester of the course, no objective testing of any 
kind was done and student grades were entirely the result of thirteen 
weekly, two-page essays. While the students responded quite favorably to 
this system, it was discovered that, for the majority of them, elimination 
of all testing meant the consequent elimination of reading. As a result, 
during the second semester, the number of papers required was reduced to 
eleven and brief reading quizzes were administered to increase motivation to 
read the text and other required materials. Format during the third semester 
of the course changed little, with the exception of a decrease in the number 
of required papers to nine. In the fourth semester, several changes were 
initiated in response to requests from a number of sources. Students asked 
that those regularly attending lecture be "rewarded" for doing so, and the 
quizzes were expanded somewhat to encompass lecture material. The teaching 
assistants' request for greater flexibility and autonomy led to a reduction 
in the number of papers to four, with length expanded to four-to-six pages 
apiece. 

With eleven individuals grading papers, the problem of reliability is, 
of course, present. During the second semester, a systematic attempt was 
made to contain the problem by insisting that ten percent of each set of 
papers be graded by another T.A. and by either the course co-ordinator or 
the administrative assistant. Because this method seemed to achieve its 
goal, it was continued during the third semester. During the fourth semester, 
with a decreased number of papers and increased T.A. autonomy to define the 
general topic for his students, this method was no longer feasible. However, 
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final grade distributions were still closely watched, as in the past, to 
maintain general similarity among the T.A. grade curves. 

Student Response to the Program 

Some of the changes introduced at different points had clear and 
immediate effects. After extending the coverage of the quizzes to include 
lecture material, lecture attendance increased by approximately thirty per- 
cent. With the reduction of the number of papers to four, T.A. role 
satisfaction clearly improved and students seemed to appreciate the decreased 
performance pressures. And, perhaps as a result of all three changes, the 
students apparently absorbed more general sociology (the increase in mean 
number correct on the general sociology test more than doubled that of the 
previous semester, from 1.9 in the third semester to 5.1 in the fourth). 

Other changes are more difficult to evaluate. Some reactions have 
remained relatively constant, independent of the varying structure and 
content of the program. Others seem to have been affected by the variations 
from semester to semester. 

The dual goal of the Purdue Experiment to develop a program that is 
both pedagogically sound while being satisfying for the student provides 
several difficulties for the evaluation of its progress. Given the nature 
of the research data, assessing the comparative adequacy of the program as 
a general learning device is quite difficult. One very tentative measure 
of the amount of sociology being absorbed by the students is the general 
sociology test. A comparison of the mean score on the "after" measure of 
this test with that of another introductory program taught in another large 
state-supported university using the same text but more conventional 



techniques provides some general indication of the relative adequacy of the 
Purdue program. The mean score for the last semester at Purdue was 25.96 
(previous semesters were 22.51, 22.07 and 21.37). The mean score in the 
other university was 27.95. At Purdue, mean increase between measures was 
approximately two for the first three semesters and five for the fourth 
semester. At the other university, it was also approximately two. While 
this is a very tentative measure, it vjould suggest that students in the 
program are absorbing comparable amounts of sociology. 

Assessing student response and evaluation of the program is less diffi- 
cult. A continuing concern of the Experiment has been student satisfaction. 
As noted earlier, courses of this size and level are stereotypically imper- 
sonal, irrelevant and often only something to be endured. The Purdue 
Experiment was, in part, an attempt to reduce or eliminate such an image. 
Therefore, the evaluations of the course and various facets of it are of 
central importance in directing the preliminary phases of the program. 

2 

Generally, student response to the course over the past three semesters 
has been quite positive. Table I indicates that two- thirds to three-quarters 
of the students like the general basis of the course and prefer to be 
evaluated on their performance in essays rather than exams. The slight 
increase in those preferring the essay system over time may reflect the 
decreasing amount of constant work demanded (eleven two-page papers were 



^While the program has been in operation for four full semesters, this 
paper reports only the last three. During the first semester, we were 
exploring the kinds of data needed and had not settled on questionnaire 
forms. Therefore, in some cases the data collected that semester is not 
comparable or not as extensive as that collected during the following semes- 
ters. Therefore, in the tables and discussion. Semester 1 will refer to 
spring, 1968; Semester 2 to fall, 1968; and Semester 3 to spring, 1969. 
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required in Semester 1, nine in Semester 2 , and four four-page papers in 
Semester 3). Clearly, students seem to prefer this method. 

The discussion sessions (termed "tutorial sessions" because of their 
emphasis on tutoring the student throu^ the course), in general (Table II), 
and in particular (Table 111), although not rated quite as hi^ly, are for 
the most part also evaluated positively. Over all three semesters, approx- 
imately half of the students saw the discussion sessions as being generally 
very valuable to them in giving understanding and help. Half also viewed 
their specific T.A. (termed "tutor") as being of considerable aid in working 
through the assignments. The sli^t decrease in high approval in both 
tables in Semester 2 may reflect several factors including the fact that 
this semester enrolled a much higher proportion of first semester freshmen 
than the other two. There were no structural changes that semester except 
for a decrease in the number of papers. The effect on over-all evaluations, 
however, was slight. 

Table IV indicates the students' reactions to what they perceive to be 
one of the most important parts of the bourse, the lecture. Again, 
approximately half rate it above average. The slight decrease in Semester 2 
is equally difficult to explain as lecture and lecture format did not change 
from Semester 1. 

The over-all response to the course as a whole is indicated in Table V. 
Here, in the students' report of their general reaction to the entire 
course, the differences become more marked. As indicated in reactions to 
the tutors and to lecture. Semester 2 was less well liked than the other 
two, with only a third ranking the course above average. In Semesters 1 
and 3, one-half and two-thirds, respectively, ranked it so. 
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Table I 

Evaluation of Essays vs. Exams 
By Percents 





Semester 1 Semester 2 


Semester 3 


Preferred Essays 


68.6 


69.3 


77.8 


Indifferent 


7.3 


6.6 


9.2 


Preferred Exams 


24.0 

(N-790) 


24.0 

(H-924) 


12.9 

(N-682) 




Table 11 






Evaluation of Tutorial 
By Percents 


Sessions 








Semester 2 


Semester 3 


Of Considerable Value 


54.0 


48.5 


59.6 


Of Some Value 


30.0 


35.2 


28.2 


Of Little or No Value 


16.1 

(N-790) 


16.3 

(N-924) 


14.9 

(N-682) 




Table III 






Evaluation of the Assigned Tutor 
By Percents 






Semester 1 


Semester 2 


Semester 3 


Of Considerable Aid 


52.7 


49.5 


59.9 


Of Some Aid 


31.7 


33.9 


32.3 


Of Little or No Aid 


16.3 

(N-790) 


16.6 

(N-924) 


7.9 

(N-682) 
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Table IV 

Evaluation of the Lecture 
By Percents 





Seneeter 1 


Semester 2 


Semester 3 


Above Average 


54.1 


46.3 


59.8 


Average 


32.3 


36.1 


32.3 


Below Average 


13.7 

(N-790) 


17.6 

(N-924) 


7.8 

(N-682) 





Table V 

Over-all Evaluation 


of the Course 






Semester 1 


Semester 2 


Semester 3 


Above Average 


50.1 


36.3 


63.0 


Average 


32.4 


39.7 


30.5 


Below Average 


17.5 

(N-790) 


24.0 

(N-924) 


6.5 

(N-682) 



o 
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Table VI 

Matrix of Association of Evaluation Items 
By Semester- -Gammas 









Evaluations 






Total 

Course 


Lecture 


Tutorial 

Svstem 


Tutor 


Essay 

Svstem 


Grade (Semester 1) 


.23 


.15 


.15 


.21 


.28 


(Semester 2) 


.23 


.15 


.15 


.17 


.24 


(Semester 3) 


.22 


.13 


.13 


.11 


.23 


Course (Semester 1) 




.68 


.38 


.43 


.56 


(Semester 2) 




.59 


.28 


.41 


.54 


(Semester 3) 




.66 


.37 


.37 


.41 


Lecture (Semester 1) 






.09 


.12 


.31 


(Semester 2) 






.03 


.11 


e 16 


(Sem€^ster 3) 






.06 


.07 


.17 


Tutorial (Semester 1) 








.82 


.28 


System (Seoiester 2) 








.81 


.23 


(Semester 3) 








.88 


.27 


Tutor (Semester 1) 










.28 


(Semester 2) 










.22 


(Semester 3) 










.22 



The set of reactions noted above suggest that the various evaluation 
measures may be highly interdependent » that reactions to one facet of the 
course may strongly effect others. Table VI presents an attempt to explore 
this possibility. This matrix of gammas indicates the association between 
the dependent variables. Interestingly » in all three semesters, grade is not 
strongly related to any of the measures and only very mildly associated with 
general evaluation of the course and approval of the essay system. The 
hypothesis of interdependence is only partially supported, however, by 
noting the rather strong to moderate associations found between the over-all 
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course rating and the other four basic evaluation items. Over all three 
semesters, total evaluation of the course is rather strongly associated with 
evaluation of the lecture and preference for the essay system and moderately 
associated with both general and specific response to the tutors and the 
tutorial system. A comparison across semesters reveals remarkable consist- 
ency with little marked variation. Except for the relationships of general 
course evaluation with other responses noted above, there are only weak 
associations between the remaining variables. The major exception to this 
is the logically strong relationship between evaluation of the tutorial 
system and evaluation of the specific tutor. Further, there is some assoc.i- 
ation between preference for the essay system and the other evaluations, 
again a logical finding. 

Clearly, then, for the purpose of evaluating the success or failure of 
the program, over-all rating of the course is, fundamentally, the most 
important dependent variable. It is the relationship of this variable with 
the others that has dominated the research interests up to this point. An 
example of some of the information resulting from this interest is indicated 
in Table VII. Using only data from this past semester. Semester 3 on the 
previous tables, associations between over-all course evaluation and other 
response and personal variables are probed by controlling on several variables 
thought to have some significance in course acceptance. While it is impossible 
in the course of this presentation to attempt to explain every discrepancy, 
some of the consistencies as well as the deviations are worthy of note. 
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Table Vll 



Over-all Course Evaluation by Other Evaluation and 
Background Variables, Controlling by Grade, Essay 
System Approval, Sex, Age, and Semester in 
College- -Gammas (except vhere 
indicated otheirwise) 



• 

Course Evaluation 
X 










Controls 










Essay System 

Grade Approval Sex 

(1) D-F (1) Prefer Essays (1) Male 

(2) C (2) Indifferent (2) Female 

(3) B (3) Prefer Exams 

(4) A 


Age 

(1) 18 or Lower 

(2) 19 

(3) 20+ 


Semester 

(1) First 

(2) Second 

(3) Third 
f4) Fourth 


Grade 






(1) 


.22 


(1) 


.16 


(1) 


.23 


(1) 


.23 








(2) 


.10 


(2) 


.27 


(2) 


.21 


(2) 


.17 








(3) 


.04 






(3) 


.22 


(3) 


.14 




















(4) 


.30 


Evaluation of 






















Lecture 


(1) 


.64 


(1) 


.65 


(1) 


.65 


(1) 


.62 


(1) 


.59 




(2) 


.65 


(2) 


.85 


(2) 


.68 


(2) 


.74 


(2) 


.69 




(3) 


.60 


(3) 


.51 






(3) 


.55 


(3) 


.68 




(4) 


.61 














(4) 


.66 


Evaluation of 


(1) 


.31 


(1) 


.31 


(1) 


.37 


(1) 


.27 


(1) 


-.12 


Tutorial System 


(2) 


.42 


(2) 


.43 


(2) 


.37 


(2) 


.41 


(2) 


.46 




(3) 


.09 


(3) 


.47 






(3) 


.47 


(3) 


.43 




(4) 


.47 














(4) 


.43 


Evaluation of 


(1) 


.43 


(1) 


.37 


(1) 


.38 


(1) 


.16 


(1) 


.00 


Tutor 


(2) 


.43 


(2) 


.28 


(2) 


.36 


(2) 


.52 


(2) 


.43 




(3) 


.12 


(3) 


.31 






(3) 


.40 


(3) 


.49 




(4) 


.28 














(4) 


.41 


Specific Tutor 
(Leik's J) 


(1) 


.17 


(1) 


.00 


(1) 


.04 


(1) 


.04 


(1) 


.06 


(2) 


.11 


(2) 


.25 


(2) 


.01 


(2) 


.01 


(2) 


.05 




(3) 


.00 


(3) 


.13 






(3) 


.04 


(3) 


.28 




(4) 


.10 














(4) 


.01 


Evaluation of 


(1) 


.11 






(1) 


.45 


(1) 


.47 


(1) 


.50 


Essay System 


(2) 


.45 






(2) 


.47 


(2) 


.42 


(2) 


.41 




(3) 


.46 










(3) 


.46 


(3) 


.23 




(4) 


.27 














(4) 


.58 


Semester in 


(1) 


.11 


(1) 


-.10 


(1) 


-.09 


(1) 


.00 






College 


(2) 


-.06 


(2) 


-.13 


(2) 


-.03 


(2) 


-.15 








(3) 


.11 


(3) 


.10 






(3) 


-.12 








(4) 


-.14 


















Age 


(1) 


.00 


(1) 


.01 


(1) 


.01 






(1) 


.23 




(2) 


.06 


(2) 


-.32 


(2) 


.04 






(2) 


.05 




(3) 


.32 


(3) 


.26 










(3) 


.00 




(4) 


.13 














(4) 


.01 


Sex 


(1) 


-.03 


(1) 


-.01 






(1) 


.01 


(1) 


-.13 




(2) 


.16 


(2) 


.35 






<2) 


.00 


(2) 


.07 




(3) 


-.19 


(3) 


.04 






(3) 


.05 


(3) 


-.31 


o 

RJC 


(4) 


.01 














(4) 


.08 
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In controlling for final grade in the course, several points emerge. 
Refreshingly, grade seems to have little to do with the evaluation items. 

It is not necessarily those who obtain high grades who like the course. 

This is especially important as it suggests that the students are making 
their evaluations independent of grade since roost of them know what grade 
they will receive at the time that evaluations are made. Surprisingly, the 
B students most deviate from the pattern. This category shows weaker associ- 
ations between general course approval and specific and general evaluations 
of the tutor as well as with the specific tutor himself. For this group 
alone, the tutor and tutorial system appear to have little to do with 
general course response. Further, this group is also atypical in the 
relationship between age and total course reaction in that only in this 
group is there any solid association between these variables. Preference for 
the essay system is most strongly associated with over-all course evaluation 
among the middle-range students, the B*s and C's, suggesting that, among 
these students, approval of the system will be an important factor in 
determining success within it. It seems likely that the most successful 
would excell under any teaching system, and therefore find the matter irrel- 
evant, while for the least successful, system approval is not sufficient 
cause for achievement. 

Even more interesting are the results obtained by controlling for 
system approval. Grade is more strongly related to general course response 
among those who prefer the essay system than among those who do not. In 
this instance, it is impossible to determine whether those who liked the 
system performed more successfully within it or vice versa. The interpreta- 
tion above would suggest that the former is more accurate, at least among 
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the B and C students. Association between course rating and evaluation of 
lecture and course rating and tutor (not the evaluation of the tutor but 
specifically which tutor he had) is strongest among those who have no 
particular preference for essays or exams reflecting perhaps that lecture 
and specific tutor become central in effecting the response to the course 
idien there is no preference for the system. Further, evaluation of the tutor 
is sli^tly less related to course evaluation among this group, suggesting 
that it is not perceived aid of the tutor that is important to course evalu- 
ation but the tutor himself. In this same group, males appear to be more 
likely to like the course, while there is no relationship among those who 
prefer one system over the other. 

Initially, it was expected that sex would make a difference in the 
evaluations of the course, as females are thought to be more verbally skilled. 
Controls on sex, however, produced virtually no differences on any of the 
items, a fact that is itself significant. 

Controls on age and semester in college (i.e., number of semesters the 
student has completed in college, including the present one) are somewhat 
difficult to evaluate because the two are so confounded. Semester by over- 
all course response controlled by age indicated that for those 18 or younger, 
semester made no difference whatsoever in approval of the course while there 
was a slight negative association for those older (i.e., the more advanced 
in school, the more they liked the course). In controls on semester, for 
first semester students, the older the students, the more they were likely 
to approve of the course while the relationship vanished for those in the 
second semester or more. Together these findings would suggest that for 
first semester students, increased age will effect course approval while for 
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more advanced students semester standing is slightly more important than age. 
Age and semester controls also uncover the fact that for younger » and even 
more, for first semester students, both specific and general evaluations of 
the tutorial system is much less associated with over-all response to the 
course than among those older or more advanced. In other words, whether 
first semester students like the tutorial system has little to do with 
whether they like the course, while for the academically more mature such 
response has quite a bit to do with course approval. 

Summary 

While it seems evident that students for the most part prefer this 
method of teaching over more conventional methods and, in general, like the 
course, there is a long way to go before the dynamics of their response may 
be understood. Reaction to the course does not appear to be strongly affected 
by structural changes. Relatively speaking, the majority of students appear 
to like the approach almost regardless of modification. What does appear to 
make some difference is the students' orientation to the essay system as well 
as independent variables like age and semester in school. 

It could be suggested that Hawthorne effect may be operating. The 
students are aware of the general focus and direction of the course as well 
as its goals. Thus, thin may present a very general positive pre-set among 
them. However, the students themselves do not experience change; no changes 
have been introduced in the middle of a semester. The students may have heard 
of changes that have been made from semester to semester but they, themselves, 
are not manipulated. Therefore, while there may be some Hawthorne effect 
operative, we do not feel that it explains observed course approval. 



o 

ERIC 



While the results of only a few of the analyses are reported here, like 
most sociological research, each table and matrix suggests several avenues 
of additional analyses. However, several tentative conclusions can be drawn 
from the information presented. First, the general evaluations would indi- 
cate that it is possible to implement a program of mass instruction that the 
majority of students would find relatively satisfactory without sacrificing 
quality in transmission of sociology. Second, given the structure employed, 
minor modifications in the system itself have little effect on the student 
response to the program. Student response seems to be affected most signif- 
icantly by other- than-structural variables. Third, of the structural 
variables, reaction to the course is most strongly associated with reaction 
to lecture, regardless of the degree to which this aspect is. underplayed. 

This is especially surprising in light of the unexpectedly weaker association 
between evaluation of the essay system, the very basis of the program, and 
general course rating. Finally, given the structure involved, first semester 
students represent a problematic group with difficulties apparently unshared 
by their more advancsd colleagues. 



